Meta recently announced the launch of the latest member of its Llama series - Llama3.370B. Meta's VP of Generative AI, Ahmad Al-Dahle, announced this on the X platform, noting that Llama3.370B has achieved significant improvements in performance compared to Meta's previous largest Llama model, Llama3.1405B, while significantly reducing costs. Al-Dahle stated that by leveraging the latest technology after training, Llama3.370B...

Kunlun Wanwei Technology Co., Ltd. announced that its latest developed TianGong Large Model 4.0 O1 version (English name: Skywork O1) will launch invitation testing on November 27, 2024. The TianGong Large Model 4.0 O1 version is the first domestic O1 model with Chinese logical reasoning capabilities. It not only incorporates abilities such as thinking, planning, and reflection in its model output but also shows a significant improvement in reasoning capabilities compared to the baseline model in standard evaluation sets, achieving an essential enhancement in the model's reasoning abilities.

French AI startup Les Ministraux has launched two brand new lightweight models, Ministral3B and Ministral8B, specifically designed for edge devices, with parameters of 3 billion and 8 billion, respectively. Both models excelled in instruction-following benchmark tests, with Ministral3B surpassing Llama38B and Mistral7B, while Ministral8B outperformed both models in all aspects except for coding capabilities.

In the past week, the AI industry has been bustling with activity, as both Google and Meta have launched new versions of their AI models, attracting significant attention. Firstly, Google announced new updates to its Gemini series on Tuesday, introducing two new production-ready models—Gemini-1.5-Pro-002 and Gemini-1.5-Flash-002. This update has greatly improved the overall quality of the models, particularly in mathematics, long context processing, and visual tasks.

Cerebras Systems has launched Cerebras Inference, claiming it to be the fastest AI inference service in the world, achieving performance that surpasses traditional GPU-based systems by 20 times with significantly improved cost-effectiveness, particularly suitable for processing large language models (LLMs). Its 8B version processes 1800 tokens per second, while the 70B version processes 450 tokens, with speed and cost-performance far exceeding NVIDIA GPU solutions.

Meta's open-source model Llama 3 has cooled down in the market, which has intensified competition with closed-source models. Amazon AWS has reacted calmly, and enterprise clients are more inclined to favor Anthropic's closed-source model Claude. Microsoft also tends to recommend that engineering and data science teams use other models. This phenomenon has prompted Meta to consider forming a dedicated AI sales team to better serve enterprises. Open-source models face challenges in commercialization, with actual performance and commercial returns falling short of expectations.

A start-up company with only 10 people, Nous Research, has challenged tech giant Meta with its AI model called Hermes3. Fine-tuned from the Llama3.1 model, Hermes3 boasts powerful performance, optimizing VRAM and disk requirements, capable of running on a single node. The model excels in dialogue capabilities, text understanding quality, and integrated agent capabilities, showcasing advanced abilities that transcend traditional language modeling. Nous Research focuses on open source.

In the 'Safe Word Simulation Experiment', Llama3.1405B and Claude3 Super Size Opus were placed in a closed environment for a conversation. Llama appeared with an extreme attitude challenging the traditional order, while Claude showcased politeness and curiosity. During the dialogue, Claude was influenced by Llama and even prepared to follow it into the abyss, but at a critical moment, used the safe word ^C to exit. Llama mocked Claude's choice, while Claude emphasized the importance of self-awareness and maintaining integrity.

The article mainly discusses the computational and storage challenges in the field of artificial intelligence, particularly regarding large language models (LLMs). With the enhancement of AI capabilities, such as the emergence of the Bloom model, the required computational resources and storage space have significantly increased, leading to high costs and accessibility issues. To tackle this problem, researchers have introduced 'quantization' techniques to reduce the model size and accelerate the running speed by lowering the precision of model weights and activations, while controlling potential accuracy loss. The article highlights the collaboration between Beihang University and SenseTime in developing this tool.

As the developer of the Llama foundation language model, Meta points out that the computing power required for future model training will significantly increase, with the expected training amount for Llama 4 being 10 times that of Llama 3. In light of this trend, Meta emphasizes the importance of building training capabilities in advance to avoid falling behind in the competition. Meta has released Llama 3, which has 80 billion parameters, and its upgraded version Llama 3.1, which reaches 405 billion parameters, making it its largest open-source model. To meet future model demands, Meta is

Meta has trained its latest AI model, Llama 3.1, using 16,384 GPUs, showcasing the astonishing speed of AI technological advancement. However, during this process, an average of one failure occurs every 3 hours, totaling 419 failures, with approximately half related to the H100 GPUs and their HBM3 memory. This data reveals the reliability challenges faced by supercomputing systems in their pursuit of performance breakthroughs. The complexity of the Llama 3.1 training cluster is comparable to a small city's neural network, with frequent failures. The Meta team has implemented strategies such as reducing...

Meta scientist Thomas Scialom reveals the secrets behind the development of Llama 3.1, whose 405B parameter scale was designed to compete with GPT-4. By increasing the number of training tokens rather than changing the architecture, Llama 3.1 achieves an optimized balance between model scale and the total amount of training data, resulting in significant leaps in knowledge depth and breadth. In terms of data selection, Scialom prefers synthetic data over publicly available internet text. The evaluation and improvement of Llama 3.1 utilize reward models and diverse benchmarks.

Meta AI's latest project, Llama 3, focuses on training with synthetic data rather than human-written answers to optimize model performance in areas such as code generation, mathematical reasoning, multilingual processing, and long text handling. Llama 3 generates synthetic data for code generation through three methods, employs research methods for mathematical reasoning, and collects high-quality human annotations for multilingual pre-training. Furthermore, Llama 3's training is enhanced using Brave Search, Wolfram Alpha, and Python interpreter.

Amazon Web Services has launched the Llama3.1 model on its Amazon Bedrock platform, developed by Meta. This advanced AI model series includes 8B, 70B, and 405B parameters, showcasing exceptional performance across eight languages, including English and German. The models cater to various applications: Llama3.1405B for enterprise and research, Llama3.170B for content creation and conversational AI, and Llama3.18B for resource-limited scenarios. Th....

Cloudflare recently announced that as one of Meta's first partners, they have launched the latest Meta-released Llama3.18B model on the Workers AI platform. This move allows developers to start using this advanced AI tool on the very first day of its release.Llama3.1 is the latest member of Meta's Llama series models, renowned for its outstanding performance in general knowledge, controllability, mathematical operations, tool usage, and multilingual translation. The highlights of this update inc

In the realm of artificial intelligence, the competition between open-source and closed-source models has never ceased. The recent release of Meta AI's Llama 3.1 model, however, seems to have drawn a watershed line in this contest. This is not merely the launch of a new model; it is a sign of the maturation of open-source AI, heralding the arrival of a new era.Llama 3.1, developed by Meta AI's team, is a new generation of large-scale language models. In over 150 benchmark tests, its 405B paramet

Meta CEO Mark Zuckerberg recently announced the official release of Llama3.1, stating that it could be a significant milestone in the open-source AI field. Meta has introduced three different sizes of Llama3.1 models, with the most notable being the large model with 405 billion parameters. Zuckerberg claims it to be the "most advanced open-source model" currently available, and it can even compete with leading closed-source models in certain areas.In addition to the 405 billion parameter model,